notebook.community

Edit and run

Retrieval of COL data in tabular format

Credits

This is a snippet for retrieving Cost of Living Data from Numbeo, available at https://www.numbeo.com/cost-of-living/

Requirements





In [65]:



    


import requests
from bs4 import BeautifulSoup
import pandas as pd



    











In [66]:



    


def get_col_data(city,currency):
    req = requests.get(f'https://www.numbeo.com/cost-of-living/in/{city}?displayCurrency={currency}')
    soup = BeautifulSoup(req.text, 'html.parser')
    mydivs = soup.findAll("table", {"class": "data_wide_table"})
    df = pd.read_html(str(mydivs), attrs={'class': 'data_wide_table'})[0]
    df['Section'] = None
    df['Section'] = df.apply(lambda x: x[0] if x[1] =='[ Edit ]' else None,axis=1)
    df['Section'] = df.Section.ffill(inplace=False)
    
    def split_range(r,pos):
        try:
            r = str.replace(str(r),u'\xa0', u'-')
            result = str(r).split('-')[pos]
        except IndexError:
            result = None
        return result
    
    df['RangeLow'] = df.apply(lambda x: split_range(x[2],0) if x[1] !='[ Edit ]' else None,axis=1)
    df['RangeHigh'] = df.apply(lambda x: split_range(x[2],1) if x[1] !='[ Edit ]' else None,axis=1)
    df['Value'] = df.apply(lambda x: split_range(x[1],0) if x[1] !='[ Edit ]' else None,axis=1)
    df['Unit'] = df.apply(lambda x: currency if x[1] !='[ Edit ]' else None,axis=1)
    
    
    df = df.drop([1,2],axis=1)
    df = df.set_index(0)
    df.loc[df.index.str.contains('%'),'Unit'] = '%'
    df.index.name = 'ProductService'
    df = df[df.Unit.notna()]
    return df

Usage

Function get_col_data with arguments city (string, e.g. 'Tokyo' and currency (3 letter code, string, e.g. 'JPY')





In [67]:



    


get_col_data('Tokyo','JPY')



    


















    
Out[67]:











  
    

      
      Section
      RangeLow
      RangeHigh
      Value
      Unit
    

    

      ProductService
      
      
      
      
      
    

  
  
    

      Meal, Inexpensive Restaurant
      Restaurants
      800.00
      1000.00
      1,000.00
      JPY
    

    

      Meal for 2 People, Mid-range Restaurant, Three-course
      Restaurants
      3000.00
      8000.00
      5,000.00
      JPY
    

    

      McMeal at McDonalds (or Equivalent Combo Meal)
      Restaurants
      600.00
      700.00
      680.00
      JPY
    

    

      Domestic Beer (0.5 liter draught)
      Restaurants
      300.00
      600.00
      400.00
      JPY
    

    

      Imported Beer (0.33 liter bottle)
      Restaurants
      300.00
      800.00
      400.00
      JPY
    

    

      Cappuccino (regular)
      Restaurants
      300.00
      500.00
      382.00
      JPY
    

    

      Coke/Pepsi (0.33 liter bottle)
      Restaurants
      100.00
      180.00
      136.67
      JPY
    

    

      Water (0.33 liter bottle)
      Restaurants
      99.00
      120.00
      104.00
      JPY
    

    

      Milk (regular), (1 liter)
      Markets
      150.00
      240.00
      186.53
      JPY
    

    

      Loaf of Fresh White Bread (500g)
      Markets
      129.00
      420.00
      209.89
      JPY
    

    

      Rice (white), (1kg)
      Markets
      400.00
      800.00
      567.65
      JPY
    

    

      Eggs (regular) (12)
      Markets
      199.00
      300.00
      225.21
      JPY
    

    

      Local Cheese (1kg)
      Markets
      800.00
      2400.00
      1,250.91
      JPY
    

    

      Chicken Breasts (Boneless, Skinless), (1kg)
      Markets
      500.00
      1500.00
      838.82
      JPY
    

    

      Beef Round (1kg) (or Equivalent Back Leg Red Meat)
      Markets
      900.00
      4000.00
      2,266.67
      JPY
    

    

      Apples (1kg)
      Markets
      500.00
      1500.00
      840.00
      JPY
    

    

      Banana (1kg)
      Markets
      250.00
      600.00
      391.43
      JPY
    

    

      Oranges (1kg)
      Markets
      398.00
      1000.00
      625.00
      JPY
    

    

      Tomato (1kg)
      Markets
      500.00
      1500.00
      792.86
      JPY
    

    

      Potato (1kg)
      Markets
      250.00
      500.00
      404.80
      JPY
    

    

      Onion (1kg)
      Markets
      200.00
      500.00
      346.50
      JPY
    

    

      Lettuce (1 head)
      Markets
      150.00
      300.00
      221.29
      JPY
    

    

      Water (1.5 liter bottle)
      Markets
      98.00
      180.00
      124.95
      JPY
    

    

      Bottle of Wine (Mid-Range)
      Markets
      1000.00
      2500.00
      1,500.00
      JPY
    

    

      Domestic Beer (0.5 liter bottle)
      Markets
      200.00
      400.00
      284.53
      JPY
    

    

      Imported Beer (0.33 liter bottle)
      Markets
      250.00
      500.00
      378.46
      JPY
    

    

      Cigarettes 20 Pack (Marlboro)
      Markets
      440.00
      480.00
      450.00
      JPY
    

    

      One-way Ticket (Local Transport)
      Transportation
      160.00
      250.00
      200.00
      JPY
    

    

      Monthly Pass (Regular Price)
      Transportation
      6000.00
      13690.00
      9,000.00
      JPY
    

    

      Taxi Start (Normal Tariff)
      Transportation
      410.00
      720.00
      470.00
      JPY
    

    

      Taxi 1km (Normal Tariff)
      Transportation
      410.00
      590.00
      410.00
      JPY
    

    

      Taxi 1hour Waiting (Normal Tariff)
      Transportation
      3085.00
      4800.00
      3,547.50
      JPY
    

    

      Gasoline (1 liter)
      Transportation
      130.00
      150.00
      139.32
      JPY
    

    

      Volkswagen Golf 1.4 90 KW Trendline (Or Equivalent New Car)
      Transportation
      2480000.00
      2850000.00
      2,500,000.00
      JPY
    

    

      Toyota Corolla 1.6l 97kW Comfort (Or Equivalent New Car)
      Transportation
      1500000.00
      2500000.00
      2,150,768.00
      JPY
    

    

      Basic (Electricity, Heating, Cooling, Water, Garbage) for 85m2 Apartment
      Utilities (Monthly)
      12000.00
      36428.57
      22,073.86
      JPY
    

    

      1 min. of Prepaid Mobile Tariff Local (No Discounts or Plans)
      Utilities (Monthly)
      15.00
      100.00
      50.15
      JPY
    

    

      Internet (60 Mbps or More, Unlimited Data, Cable/ADSL)
      Utilities (Monthly)
      3300.00
      6000.00
      4,684.92
      JPY
    

    

      Fitness Club, Monthly Fee for 1 Adult
      Sports And Leisure
      8000.00
      13000.00
      9,789.47
      JPY
    

    

      Tennis Court Rent (1 Hour on Weekend)
      Sports And Leisure
      1500.00
      4000.00
      2,492.31
      JPY
    

    

      Cinema, International Release, 1 Seat
      Sports And Leisure
      1800.00
      2000.00
      1,800.00
      JPY
    

    

      Preschool (or Kindergarten), Full Day, Private, Monthly for 1 Child
      Childcare
      30000.00
      75000.00
      39,464.29
      JPY
    

    

      International Primary School, Yearly for 1 Child
      Childcare
      1500000.00
      2500000.00
      2,173,333.33
      JPY
    

    

      1 Pair of Jeans (Levis 501 Or Similar)
      Clothing And Shoes
      3500.00
      10000.00
      6,471.43
      JPY
    

    

      1 Summer Dress in a Chain Store (Zara, H&M, ...)
      Clothing And Shoes
      2160.00
      7000.00
      4,365.93
      JPY
    

    

      1 Pair of Nike Running Shoes (Mid-Range)
      Clothing And Shoes
      6000.00
      11000.00
      7,928.12
      JPY
    

    

      1 Pair of Men Leather Business Shoes
      Clothing And Shoes
      7000.00
      20000.00
      11,531.25
      JPY
    

    

      Apartment (1 bedroom) in City Centre
      Rent Per Month
      85000.00
      170000.00
      119,066.67
      JPY
    

    

      Apartment (1 bedroom) Outside of Centre
      Rent Per Month
      50000.00
      120000.00
      75,512.20
      JPY
    

    

      Apartment (3 bedrooms) in City Centre
      Rent Per Month
      200000.00
      400000.00
      263,333.33
      JPY
    

    

      Apartment (3 bedrooms) Outside of Centre
      Rent Per Month
      100000.00
      200000.00
      143,172.41
      JPY
    

    

      Price per Square Meter to Buy Apartment in City Centre
      Buy Apartment Price
      1000000.00
      1500000.00
      1,166,666.67
      JPY
    

    

      Price per Square Meter to Buy Apartment Outside of Centre
      Buy Apartment Price
      400000.00
      675000.00
      539,791.62
      JPY
    

    

      Average Monthly Net Salary (After Tax)
      Salaries And Financing
      nan
      None
      325,612.53
      JPY
    

    

      Mortgage Interest Rate in Percentages (%), Yearly, for 20 Years Fixed-Rate
      Salaries And Financing
      0.90
      2.00
      1.20
      %

Content source: cnstlungu/Notebooks

Similar notebooks:

notebook.community | gallery | about